Video to CT Registration for Image Overlay on Solid Organs

This paper describes a general-purpose system for computing registered stereoscopic video overlays of pre-operative imagery during minimally invasive surgery. There are three key elements to our approach. The first element is a real-time computer vision system that operates on stereoscopic video acquired during minimally invasive surgery to extract geometric information. We present two variations on this system: a dense stereo algorithm and a sparse point-based method. The second element is an efficient deformable surface-to-surface ICP registration. The final element is a novel display system that has been customized to operate well with stereo vision. By combining these elements, we show that we are able to perform video to volume registration and display in real time. This in turn facilitates rendering of annotations and visualization of sub-surface information on structures within the surgical field. Experimental results are shown on video sequences recorded during animal and human surgeries.